A Bulk-Parallel Priority Queue in External Memory with STXXL
نویسندگان
چکیده
We propose the design and an implementation of a bulkparallel external memory priority queue to take advantage of both sharedmemory parallelism and high external memory transfer speeds to parallel disks. To achieve higher performance by decoupling item insertions and extractions, we offer two parallelization interfaces: one using “bulk” sequences, the other by defining “limit” items. In the design, we discuss how to parallelize insertions using multiple heaps, and how to calculate a dynamic prediction sequence to prefetch blocks and apply parallel multiway merge for extraction. Our experimental results show that in the selected benchmarks the priority queue reaches 64% of the full parallel I/O bandwidth of SSDs and 49% of rotational disks, or the speed of sorting in external memory when bounded by computation.
منابع مشابه
Parallel Priority Queue and List Contraction: The BSP Approach
In this paper we present eecient and practical extensions of the randomized Parallel Priority Queue (PPQ) algorithms of Ranade et al., and eecient randomized and deterministic algorithms for the problem of list contraction on the Bulk-Synchronous Parallel (BSP) model. We also present an experimental study of their performance. We show that our algorithms are communication eecient and achieve sm...
متن کاملThe Contention Avoiding Concurrent Priority Queue
Efficient and scalable concurrent priority queues are crucial for the performance of many multicore applications, e.g. for task scheduling and the parallelization of various algorithms. Linearizable concurrent priority queues with traditional semantics suffer from an inherent sequential bottleneck in the head of the queue. This bottleneck is the motivation for some recently proposed priority qu...
متن کاملEquivalence between Priority Queues and Sorting in External Memory
A priority queue is a fundamental data structure that maintains a dynamic ordered set of keys and supports the followig basic operations: insertion of a key, deletion of a key, and finding the smallest key. The complexity of the priority queue is closely related to that of sorting: A priority queue can be used to implement a sorting algorithm trivially. Thorup [11] proved that the converse is a...
متن کاملA new multi-mode and multi-product hub covering problem: A priority M/M/c queue approach
One main group of a transportation network is a discrete hub covering problem that seeks to minimize the total transportation cost. This paper presents a multi-product and multi-mode hub covering model, in which the transportation time depends on travelling mode between each pair of hubs. Indeed, the nature of products is considered different and hub capacity constraint is also applied. Due to ...
متن کاملCombining the Sweep-Line Method with the Use of an External-Memory Priority Queue
The sweep-line method is an explicit-state model checking technique that uses a notion of progress to delete states from internal memory during state space exploration and thereby reduce peak memory usage. The sweep-line algorithm relies on the use of a priority queue where the progress value assigned to a state determines the priority of the state. In earlier implementations of the sweep-line ...
متن کامل